feat: coordinate concurrent CBM sessions - #1139
Merged
Merged
Conversation
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…ion-daemon Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> # Conflicts: # src/mcp/mcp.c # tests/test_cli.c
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> # Conflicts: # Makefile.cbm
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
DeusData
added
enhancement
DeusData
added
editor/integration
The Unix-VFS MAX_PATHNAME bump from 512 to 4096 shipped with the daemon work but left no paper trail, so the next amalgamation refresh would silently revert it. Document it in vendored/sqlite3/PATCHES.md with the rationale and re-apply procedure, and point to it from THIRD_PARTY.md. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The six suites that spawn coordinated worker subprocesses or bind local endpoints (index_supervisor, daemon_application, daemon_runtime, daemon_frontend, daemon_bootstrap, daemon_ipc) ran in the parallel wave, where the saturated 3-4-core CI runners starve their fixed readiness deadlines: index_supervisor waits at most 3 s for a worker marker while the worker is a full re-exec of the ASan runner plus the admission handshake. Both macOS legs failed the same four readiness assertions deterministically and ubuntu-latest added four daemon_frontend and one daemon_runtime timing failures, while an idle machine passes the same suites 6/6 in the wave. Moving them to the quiet serial tail follows the runner's existing rule for this class (cli/subprocess/watcher/...) and also keeps the shared per-account coordination namespace free of cross-suite admission traffic. Same suites, same tests, same union-guarded totals - only the schedule changes. Locally re-verified: 6742 passed, 0 failed, 2 skipped (120 suites, 17 serial-tail). Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The launcher answered every ownership/ACL problem with one generic "launcher ownership or access policy is unsafe", which made the CI guard failures (and any field report) undiagnosable: the same message covers an untrusted owner SID, a mutation-capable ACE, a reparse point, a hard-link clone, and a plain open failure across the whole ancestry walk. Record the failing check at its site - offending path, owner or ACE SID in SDDL string form, rights mask, Win32 error - and print it once alongside the refusal. No policy changes; refusals stay refusals. The interactive MCP smoke now also dumps daemon-conflicts.ndjson next to the daemon log tail on failure, so an admission conflict names itself instead of surfacing as a bare 30 s client timeout. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…guards Completes the handoff task for the Windows harness: the native test job ran with the MSYS-shared /tmp and the runner's inherited LocalAppData\Temp ACLs, which the daemon's strict private-filesystem checks reject by design. The test-windows job now creates a per-user root under the profile with an owner-stamped, protected current-SID DACL and routes TEMP/TMP (native form) and TMPDIR (POSIX form) through it before scripts/test.sh runs. The guards script already hardened its root DACL but ownership is never inherited on Windows: bundle copies created by the admin-group runner token can come out Administrators-owned, and the launcher's exe policy requires the exact current user as owner. Stamp the current SID on the staged bundle dir and both executables explicitly. Job topology, gates, and triggers are unchanged; the new step is a sub-second pwsh block in the existing test-windows job. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…bjects The Windows SECURITY_ATTRIBUTES used for every private-namespace creation (runtime dirs, rendezvous/startup/participant/lifetime/cohort lock files, log files, pipes) carried a DACL but no owner, so created objects took the token DEFAULT owner. Admin-group tokens can default that to BUILTIN\Administrators - standard policy on Windows Server, including CI runners - while every strict validation gate (win_file_owner_secure(require_current_user=true), private_win_owner_only_dacl) demands the exact token-user SID. Result: the daemon rejected objects it had just created itself; the client refused the daemon's rendezvous record, retried silently for its full 30 s window, and each clientless generation clean-stopped, looping until the frontend gave up. The legacy kernel mutex was the one object that already force-set its owner post-create - it is the template this change generalizes. Both descriptor builders now set the owner explicitly at creation, so objects are born satisfying the same strict gates on every default- owner policy. The gates themselves are unchanged. The Windows client connect path also stops failing mute: terminal rendezvous refusals, deadline expiry (with last status), and pipe server-identity rejections now log their reason once, matching the launcher-side refusal diagnostics. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Stamping the owner at creation covers objects born through the daemon's own security descriptors, but the cache root is created by plain mkdir_p before the gate runs, and harness- or user-created directories arrive the same way. On an Administrators-default-owner token those directories are born owned by BUILTIN\Administrators and the exact-user gate refused them permanently - including a real elevated admin's own cache directory. win_runtime_directory_secure (the final component of both the runtime dir and the cache-root validation) now treats a TRUSTED owner - the same SYSTEM/Administrators/TrustedInstaller set the launcher already accepts for directories - as repairable: the owner is re-stamped to the exact token user inside the same set_security_info call that already re-protects the DACL. Any other owner is refused exactly as before, the post-repair validation still demands the exact user, and the handle falls back to the old access mask when WRITE_OWNER is unavailable so no previously-passing configuration can newly fail. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The new pipe_rejected diagnostic proved the Windows client reaches the daemon's pipe and then refuses the SERVER identity for its whole 30 s window, while earlier sessions in the same run connect fine. The check compares the server-process token user, so the interesting failures are environmental: a dead server PID already reused by Windows (typically for a SYSTEM service) is indistinguishable from a hostile pipe without classification. The rejection now names the failing step (pid query, process open, token open/query) or the server SID class (system/admins/other) plus the server PID. Also reflows the two owner-stamping call sites the lint gate flagged. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…l legs The local containers ran with every host core, which masks the exact failure class the 4-core GitHub runners catch: deadline/starvation misses in worker-readiness waits and watcher/subprocess deadlines. The test services now default to a 4-CPU cap (CBM_LOCAL_CI_CPUS overrides, 0 = unconstrained), so a local run exercises the same scheduling pressure as CI before CI ever sees the push. Compilation is cached through ccache in persistent named volumes with CCACHE_COMPILERCHECK=content: every hit is keyed on the compiler-binary content plus the fully preprocessed input, so a stale or foreign entry can only miss, never return wrong output. Also documents the Wine leg's honest limit: it compile-checks Windows but cannot reproduce real ACL/token/owner semantics. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…lity mode Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
cli_windows_prepare_install_pair folded launcher staging, payload staging, the payload fingerprint match, and the staged-launcher --version probe into one bool, so the caller could only report "launcher/payload source pair did not remain runnable after private staging" without saying which sub-step failed. Windows cbm_exec_no_shell does not inherit the child's stderr, so the launcher's own refusal reason never reaches the log either. Split the checks so each failing sub-step prints a distinct error, turning the opaque Phase 8 smoke failure into a precise signal on the next run. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
activation_windows_security_init built the creation security descriptor with a protected current-user DACL but never set an owner, so a file created with these attributes inherited the token's default owner. Under an Administrators-default-owner policy -- standard on Windows Server and on GitHub's elevated Windows runners -- that default is BUILTIN\Administrators, not the calling user. Every staged file is then validated with the strict activation_windows_owner_is_current check, which rejected the Administrators-owned file and made private staging fail with "launcher/payload source pair did not remain runnable after private staging" (smoke Phase 8). Stamp the owner explicitly to the token user in the creation descriptor, the same fix the daemon side already applies. This keeps the strict owner check intact -- the staged file genuinely becomes current-user-owned -- so it is not a validation relaxation; setting the owner to our own token user needs no extra privilege. Complements the launcher's trusted-owner acceptance, which covers the installed binary that a non-admin user cannot re-stamp. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The Phase 8 staging diagnostic added two long lines that clang-format-20 (the CI formatter) wraps differently. Reformat them so lint passes; no behavior change. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…iptor cbm_mkdtemp on Windows created the directory with plain _mkdir, so it took the token's default owner and the parent's inheritable DACL. Under an Administrators-default-owner policy -- standard on Windows Server and GitHub's elevated runners -- the temp directory was born owned by BUILTIN\Administrators with foreign inherited grants. The activation transaction validates its staging directory with a strict owner-is-current- user check plus a trusted-only DACL scan (activation_directory_secure), so install staging failed at the directory gate before any file was staged: "private staging failed for the launcher" in the Phase 8 smoke, previously masked as "did not remain runnable after private staging". Create the directory with CreateDirectoryW and an explicit descriptor: owner stamped to the token user and a protected, inheritable, user-only DACL -- private by construction, the same policy the daemon applies to its runtime directory. If the descriptor cannot be built the code falls back to plain _mkdir so degraded environments (Wine) keep working; downstream validation still gates security there. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The Windows VM becomes a first-class, reproducible local-CI leg driven entirely over the local ssh link by maintained scripts: - provision-windows.sh: idempotent host-side provisioner — msys2 base, CLANGARM64 (native, fast) + CLANG64 (CI arch) toolchains, repo clone/update, full build, smoke check. One command recovers a fresh VM or a lost disk. - win.sh: daily driver — status/update/build/test/guards/smoke-install/ sh/push-file/asan-build. smoke-install runs the managed-install E2E with stderr visible, the class the CI smoke probe structurally hides. - windows-bootstrap.ps1: reduced to the one-time in-VM door-opener (OpenSSH + host key + GitHub-runner default-owner policy mirror), with the ISO delivery trick documented; toolchain work moved to the host-side provisioner. The previous winget path does not work over ssh and clipboard/shared folders do not exist on Windows-ARM guests. - README.md: full lifecycle — VM creation, clone-before-drivers rollback point, SPICE guest tools, bootstrap, provisioning, daily loop, and the honest limits (no ASan on aarch64-windows; ASan also faults under x64-on-ARM emulation, so Windows sanitizer coverage stays on CI). Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The activation transaction and the release-descriptor probe validate the
full ancestor chain of everything they stage: every path component must
have a trusted owner and grant mutation rights only to trusted identities.
That walk is deliberate TOCTOU defense and stays strict. The staging
directory, however, came from the process temp root, which routinely lives
in a tree that legitimately fails the walk through no fault of ours: under
msys2 TMP resolves inside the msys install tree (C:\msys64\tmp), and on
GitHub runners inside the runner work directory, both of which carry broad
inherited grants such as Authenticated Users modify on an upper component.
The staged pair then failed validation ("launcher descriptor candidate is
unsafe" / the smoke Phase 8 --version probe refusal) even though the staged
files themselves were private.
Add cli_windows_staging_root(): stage install and update transactions under
%LOCALAPPDATA%\codebase-memory-mcp, whose ancestor chain is user-owned end
to end, so the strict walk passes by construction instead of by environment
luck. Falls back to USERPROFILE\AppData\Local and finally the process temp
root when the environment is degraded.
Also make the staging observable where it previously failed blind:
cli_windows_stage_private_file reports which transaction step failed with
the OS error code, and cbm_mkdtemp warns once if the private security
descriptor cannot be applied and it falls back to default directory
security. A blind boolean here cost several blind CI round-trips.
win.sh: scp targets need native C:/ paths on Windows OpenSSH.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Back-to-back requests on one frontend<->daemon connection failed roughly half the time on loaded Windows hosts, surfacing as wandering MCP session drops (smoke Phase 5/5e/5f "no tools/list response", "timed out waiting for id=2"). Root cause, confirmed on real Windows: the application worker stored its done flag AFTER writing the response, so a client that pipelines its next request the instant it sees a response raced the previous worker's final instructions and its join, and admission rejected the new request with BUSY. On EOF that BUSY was then classified as an expected shutdown and the response was silently dropped with a success exit; mid-session it exited non-zero. A timing-sensitive heisenbug — any added stderr shifted the window and it passed. - runtime_application_worker publishes the done flag BEFORE sending the response, so completion is one atomic transition from the client's view: by the time any client can react, the slot already reads reusable. The admission side joins the thread after observing the flag, which absorbs the send tail. No timing constants. - frontend_should_stop's input-closed leg now also requires no request in flight, so a daemon-side failure of the final post-EOF item is loud, never swallowed as an expected stop. - The EOF drain is now progress-aware: it resets its window on each completed item, bounding stuckness rather than total batch length, so a slow cold daemon spawn no longer abandons queued tail responses. - Latch the never-connected shutdown window on a monotonic admitted-total counter instead of sampling the live client count, which can read zero between two short-lived sessions and stop a daemon mid-conversation. - Add structured lifecycle diagnostics (daemon.runtime_stopping reason=, lifetime_end reason=, connection_end) so this class is observable in the logs instead of being inferred. Validated: real-Windows loop 12/12 (was ~50% loss), macOS daemon suites 143/143. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…dget The backpressure/EOF fixtures waited only 10s for the forked daemon to announce readiness, but the daemon's own startup includes the cohort-claim path whose internal budget is HOST_DAEMON_CLAIM_TIMEOUT_MS (12s). Under the 4-core oversubscription of CI runners a slow-but-legitimate startup blew past 10s and was SIGKILLed before it could announce, so daemon_frontend backpressure failed almost every ubuntu run while passing on fast machines. Raise the announce deadline to 30s, comfortably above the daemon's own worst-case budget. Verified: the backpressure failures no longer reproduce under a local CI-fidelity 4-core run. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
… chain
cli_windows_staging_root builds the per-user install staging root from
LOCALAPPDATA/USERPROFILE, but msys and CI harnesses export those in POSIX
drive form ("/c/Users/..."), which the CRT cannot resolve, and the target's
AppData\Local ancestors may not exist yet on a fake-profile fixture. Rewrite
a leading "/c/" to native "C:/" before use, create the whole ancestor chain
(a single mkdir left the root missing), and fall back to the temp root only
after verifying the directory exists. Also name the failing sub-step when
private staging cannot even create its directory, so the opaque "did not
remain runnable" message is no longer the only signal.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…llide scripts/build.sh, test.sh, and clean.sh hardcoded build/c, and clean.sh rm -rf'd it unconditionally. Running the container legs concurrently with a native macOS build then had them delete and relink each other's objects mid-build (ld "file cannot be opened"). Thread a BUILD_DIR through the three scripts (default build/c) and point every compose service at its own dir (build/linux-arm64, build/linux-amd64, build/win-cross, build/alpine); native macOS keeps build/c exclusively. Also run the lint service in CI-parity mode (--ci: cppcheck + clang-format only), matching CI, since clang-tidy is in neither the lint image nor the CI job. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Add vm-smoke.sh, which stages the release pair (launcher + payload) in a profile-rooted temp dir and runs scripts/smoke-test.sh exactly as the CI windows pr-smoke step does, so the real-Windows leg exercises the same managed-install path. Extend win.sh with ubsan-build/ubsan-test (UBSan works under x64-on-ARM emulation; ASan does not), a PageHeap toggle for OS-level heap checking of the native runner, and a full-suite parallel run. Document the honest sanitizer limits and the daily loop in the README. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Owner
Author
Windows daemon/install state — findings & remaining workHandoff notes after a deep real-Windows debugging pass. A UTM Windows‑11‑ARM64 VM was used as a local leg (driven over ssh) to see stderr/daemon logs that CI structurally hides. Commits pushed on this branch; the one open item is called out at the end. Fixed & validated
Open item — mapped‑image capability probe (Phase 8)With staging fixed, the VM smoke now reaches the managed‑install E2E and fails at the launcher capability probe: It fails at Other findings worth a look
Local Windows harness
|
Owner
Author
Handoff to Claude — daemon v1 local worktree (2026-07-19)Work was stopped immediately at the user's request. All workers are stopped. Nothing in this handoff was committed, pushed, or merged. Critical state: preserve this worktree
Product scope remains narrowFinish the mandatory per-account coordination daemon v1 and make this PR green. Do not add downstream features. CLI one-shot commands remain separate/local; long-lived MCP/hook servers use the daemon. No merge is authorized in this handoff. Infrastructure is operationalThis is not blocked by the Codex sandbox. Host socket/VM commands merely need the escalated execution path in Codex.
Implemented locally and already proven at least once1. Windows
|
The Windows-VM bootstrap embedded a personal SSH public key and the drivers connected with StrictHostKeyChecking=no. Key material and host identity are now strictly operator-supplied and live outside the repository (~/.claude/cbm-vm/, never committed): - windows-bootstrap.ps1 takes the operator's public key as a mandatory -SshPublicKeyPath argument (validated single Ed25519 line) and prints the VM host-key fingerprint to pin. Generating your own key is documented in the script and README: ssh-keygen -t ed25519 -N "" -C <label> -f ~/.claude/cbm-vm/id_ed25519 - win.sh and provision-windows.sh verify the pinned CBM_VM_HOST_KEY_SHA256 fingerprint before every connection (StrictHostKeyChecking=yes against a keyscanned known_hosts, Ed25519 only, IdentitiesOnly), validate branch names before remote interpolation, and fail remote pipelines loudly (set -e -o pipefail). - The MSYS2 installer download is pinned to a versioned release URL with SHA-256 verification instead of an unpinned latest. - win.sh gains a sync subcommand mirroring the uncommitted worktree onto the VM; its manifest logic is guarded by tests/test_vm_worktree_manifest.sh, wired into scripts/test.sh. guards now runs the maintained PowerShell driver against a clean product build; smoke-install drives vm-smoke.sh. - run.sh drops the stale duplicate windows-vm/mac-vm legs (the former carried its own unpinned ssh path) in favor of the vm/ drivers. The previously committed public key has been rotated off the VM and is no longer authorized anywhere. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…rdening, long-path launcher transactions Daemon lifecycle and Windows correctness, verified on a real Windows 11 ARM64 VM through the maintained test-infrastructure/vm drivers, plus the macOS and Linux arm64 suites and the container lint gate. Daemon lifecycle: - daemon start/stop/status subcommands. `daemon start` launches a PERMANENT daemon (spawn shape is byte-exact argv; survives idle periods and session ends) and reports an already-active daemon instead of failing. Permanence is honored at every stop latch: last-committed-client disconnect, host initial-client window, coordinator release, and application final-session close — a permanent daemon also keeps admitting new sessions after its last one closes. - daemon stop refuses while sessions are active and lists the blocking peers (pid/role) that must finish first; an idle daemon drains through the activation-shutdown machinery with the ACK ordered after connection interrupts. A second stop is idempotent. The wire ops are no-cohort first-frame requests with peer fingerprint authentication, so stop/status never conflict with an exact-build admission gate. - One-shot CLI commands now execute through the daemon (index workers keep their local supervised path). A cold CLI run that had to spawn a temporary daemon prints a hint that `daemon start` removes the per-command startup tax; a warm daemon is recycled silently. - Hooks are connect-only fail-open: with no daemon present the hook emits a visible, rate-limited notice (Claude-dialect systemMessage plus stderr for other dialects) and always exits 0 — augmentation is never allowed to block the caller's tool use. - Version skew: a newer-build client automatically drains an older-build permanent daemon (strict semantic-version triples only; dev builds never auto-drain) and the build-conflict message names `cbm daemon stop` as the manual escape hatch. Windows IPC/runtime (real-VM verified): - ipc(win): persistent pending overlapped ConnectNamedPipe. The accept path used to destroy its listening pipe instance on every 20 ms poll timeout; a client attaching in the teardown window was severed or left on an orphaned pipe object whose HELLO no server handle could ever read, absorbing the connect until the client's own timeout expired. The pending connect now survives poll timeouts and nothing is destroyed while a client could be attaching. - ipc(win): drain-before-close for final responses. Closing a named-pipe server handle can discard a just-sent response before the peer reads it (POSIX stream sockets never lose buffered data on close). A bounded cbm_daemon_ipc_connection_drain (read-until-peer-EOF; no-op on POSIX, immediate on interrupted connections) now precedes close in runtime_worker_finish and runtime_reject_inline, so hello-conflict, capacity and disconnect acknowledgements reliably reach the peer. - runtime: CLOSE_INTENT wire frame. A Windows named-pipe client has no transport half-close, so close_begin now announces departure with an explicit frame (ordered after APPLICATION_CANCEL, before the local interrupt); the server releases the client's admission on receipt instead of waiting for the handle to close. Admission-drop timing is now identical to POSIX shutdown() semantics on every platform. - runtime(win): client close cancellation. close_begin serializes with request publication under the send lock, best-effort sends the active token's APPLICATION_CANCEL frame, then interrupts local I/O; the server cancels MCP/subprocess work promptly. Contract tests accept both correct outcomes (interrupted transport or decoded CANCELLED). - runtime: activation acknowledgement ordering. The activation ACK is the requester's license to act on "snapshotted and draining", so every connection interrupt is now initiated before the ACK is sent; a session could previously get one more request serviced after the requester observed the ACK. - service(win): deadline-bounded private-file prepare. The conflict-log prepare retry loop (100 x Sleep(2), which rounds up to the ~16 ms timer granularity) burned ~1.6 s against permanently obstructed paths, stalling hello rejections past the client's timeout. The retry budget is now a 250 ms deadline; transient share collisions still retry. - subprocess(win): cmd.exe /C payload encoder quotes metacharacters correctly (root cause of the git-on-Windows failure cluster). - watcher: SHA-256 buffer sizing (CBM_SZ_64 -> CBM_SZ_128) and a native Windows stop/unwatch cancellation test with exact-image verification. - httpd: send_all writes in bounded 64 KiB slices. A single giant nonblocking send() on Windows is absorbed wholesale into AFD kernel buffering regardless of SO_SNDBUF, so send deadlines and interrupts could never engage against a slow peer (and the full payload was pinned in nonpaged pool). Slicing restores a deterministic backpressure point; a test hook pins SO_SNDBUF for the deadline and interrupt tests. - ui/http: shutdown lifecycle — interrupt checks, response-wide send deadline, explicit connection states, refusal to free a server while a listener-owned connection is active. Windows long-path support: - Central path-aware wide conversion (canonicalize via GetFullPathNameW and prepend the extended-length prefix for absolute paths >=240) at the compat chokepoints (cbm_fopen/compat_fs/mkstemp/mkdtemp), sqlite store opens, and the daemon build-fingerprint/log paths. Deep managed installs (a 64-hex generation directory routinely exceeds MAX_PATH) now index, stage and activate correctly. - activation transaction: its own file APIs and the component-walking ancestry validators now operate in the extended-length namespace; the launcher path is canonicalized (and prefixed when deep) once at entry so every downstream exact-string comparison stays form-consistent. - Executable self-resolution uses the wide APIs (GetModuleFileNameW, GetFileAttributesW) so non-ASCII install paths survive argv[0] resolution. Windows launcher install/uninstall transaction: - FileRenameInfoEx names are NUL-terminated in an over-allocated buffer. FileNameLength governs per the contract, but filter drivers read FileName as NUL-terminated and appended adjacent heap bytes to created names — a flaky, garbage-suffixed rename target. Both the CLI and the launcher rename helpers are fixed. - Uninstall retires state via rename-aside (.cbm -> .cbm-retired-v1-<tag>-<pid>) with the retired tag shortened to 16 hex chars so the bare rename target stays under the FileRenameInfoEx NT-conversion ceiling at guard depths; 64 bits still uniquely identify the generation. - When the running launcher's mapped generation backings pin .cbm against rename, the backings are relocated to activation-<pid>-N .retired tombstones beside the install (a mapped image may be renamed, never deleted; the launcher's liveness-guarded sweep reclaims stale tombstones). Every relocation is recorded, and a FAILED uninstall reverses the moves after restoring .cbm — via MoveFileExW with extended-length paths on both arguments, since the deep generation target is beyond the handle-based rename's bare-path reach — so a restored install keeps its generation backings and stays runnable. - After a committed uninstall the retired tree's backings are relocated out so the tree is shallow enough for the detached cleanup's rd, and the cleanup's working directory strips the extended-length prefix (CreateProcessW lpCurrentDirectory silently ignores prefixed paths). - Files created under Administrators-default-owner directories (CopyFileW destinations, CREATE_NEW tombstones, probe directories) are explicitly owner-stamped so the exact-owner validators hold on runner images; guard fixtures stamp hand-built trees the same way. Diagnostics, tests and infra: - diagnostics: discovery is now an always-delivered JSON control record (new cbm_log_control) that survives CBM_LOG_LEVEL suppression and paths containing spaces; placement honors $TMPDIR with /tmp fallback via a diagnostics-local helper; the soak parser reads the JSON record; documented in docs/CONFIGURATION.md. Red-first coverage for suppressed log levels, TMPDIR-with-spaces, and native Windows output-contract assertions. - tests(win): daemon_ipc/daemon_frontend fixtures now build endpoint parents with production-shaped ancestry (LocalAppData on Windows, via th_secure_runtime_parent_new) — the runtime ancestry validation correctly refuses temp roots whose ancestors grant mutation rights to Authenticated Users (C:/msys64/tmp, GitHub-runner work dirs) — and drive the documented startup-owner publication flow before reading generation-bound endpoint addresses. This turns the 26 Windows failures previously visible in CI's full-test job green without weakening any validation. - tests(win): the launcher guard covers the full permanent-launcher contract including failed-uninstall restore and immediate reinstall after uninstall; new daemon lifecycle and reworked hook-augment guards run the start/recycle/stop flow end to end. - tests: CBM_SKIP_PERF is now actually consumed by the test runner (it was set by CI but never read, so perf suites ran everywhere); four throughput/bench suites are classified as perf, the heavy store_arch suite moved to the slow-timeout tier, and two wall-clock-sensitive assertions were rewritten as invariant checks with coarse hang-detector backstops. - build/test infra: build-dir safety contract, UI dev-proxy security contract, soak daemon-recovery contract, path-safety helper, the Windows VM worktree-sync contract wired into scripts/test.sh, and vm/win.sh guards building its clean embedded-UI product in an isolated BUILD_DIR so it cannot clobber the incremental test build. provision-windows.sh now installs Node.js for the guards UI build. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…resolution Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Fixes surfaced while bringing the shared-coordination-daemon branch green across macOS, the Linux containers, and the real Windows-ARM64 VM, plus a new stability guard that caught one of them. - daemon(win): the CLI teardown transition latched a failed release as permanent, but a Windows participant-state release must briefly try-hold the shared startup/legacy gates and legitimately collides with a concurrent one-shot's teardown; the release is retriable by contract (it always retains the transition), so parallel one-shot commands no longer report "CLI coordination cleanup failed" despite succeeding. Found by the new stability guard's churn section. - cli/main(win): install/update/uninstall and the Augment hook-script removal derive the managed launcher's identity from its plain drive-form path (cli_windows_plain_utf8), not the \\?\ extended-length form, so agent-config ownership matches on Windows and uninstall no longer leaves the MCP entry or owned hook scripts behind. - security: the `daemon start --open` browser launch is now shell-free (ShellExecuteW on Windows, cbm_exec_no_shell elsewhere) instead of system(); the CORS origin check spells out its two literal loopback URLs so the static URL audit sees a complete value. Both clear the Layer-1 allow-list audit. - coordination: version-cohort lock retries now sleep a per-process jittered interval — fixed-period retries can phase-lock two participants so one starves. The activation-quiesce test's observation window now covers worst-case candidate staging (three tamper-defense hashes over a ~1 GB sanitizer binary exceed the old 30 s budget on container I/O). - smoke/soak: the agent-config smoke matches Windows config paths through their escaped-backslash quoted form; the soak parser skips the update-available banner before the JSON summary and gained a one-shot CLI admission-churn phase inside the RSS/FD leak window. The build-dir safety fixture creates a real symlink/junction (MSYS2 ln -s otherwise deep-copies) so its traversal-refusal contract is genuinely exercised, and the symlinked-agent-roots test demotes its planted links to an unprivileged owner when run as root so the refusal is really tested. - tests(win): a new daemon-stability guard exercises the parameter surface, hook fail-open with its rate-limited notice, start-twice and occupied-port handling, busy-stop refusal against a live MCP session, kill -9 crash recovery, and sequential/parallel one-shot churn. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Close the cross-platform sanitizer gaps that were leaving real concurrency and undefined-behavior bugs uncaught, and fix a data race the first widened run surfaced. - tsan: the data-race gate ran three suites (mem, slab_alloc, parallel) over no real threaded production code. It now covers every threaded surface that runs clean and stable under TSan: the parallel-extraction worker pool (parallel, worker_pool, pipeline), the filesystem watcher, the embedded HTTP server (httpd), diagnostics sampling, the MCP server and mutation guard, subprocess supervision, and the runnable daemon-coordination paths (daemon, daemon_application). daemon_runtime (deadlocks under TSan+fork), daemon_ipc and daemon_frontend (test-harness synchronization, not production) are excluded with the reasons recorded in the Makefile. - tsan: fixed a genuine data race the widened gate immediately found — cbm_lsp_max_walk_depth's lazy cache was read and written by parallel LSP-extraction workers without synchronization. A data race is undefined behavior even when every worker computes the same value, so the cache slot is now a relaxed atomic: a plain load on the hot path, and a first-touch double-compute simply stores the same value. - tsan(ci + local): the test-tsan job now runs on Linux amd64, Linux arm64, AND native ARM64 macOS (the threading code is shared, so a race is usually caught on all three, but scheduler differences let each surface one the others miss). The local ladder gained `run.sh tsan` and `tsan-amd64` plus the matching compose services. TSan's shadow memory aborts under modern high-entropy ASLR, so the containers run under `setarch -R` with an unconfined seccomp profile (the personality syscall is otherwise blocked) and the CI Linux legs lower vm.mmap_rnd_bits first; amd64 TSan cannot run under x86_64-on-ARM translation and is a real-hardware/CI gate only (documented in run.sh). - ubsan(win/arm64): native ARM64 Windows had no sanitizer at all — AddressSanitizer ships no aarch64-w64-windows-gnu runtime. UBSan in trap mode (-fsanitize-trap=undefined) needs no runtime library, so it instruments natively and turns undefined behavior into an illegal-instruction trap; -fstack-protector-strong adds stack-smash coverage the heap tools miss. The GitHub windows-11-arm leg switches from unsanitized to this, and vm/win.sh gains trap-ubsan-build / trap-ubsan-test for local iteration (reproduce under the emulated x86_64 UBSan to see which check fired). The whole codebase builds and runs clean under it. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
DeusData
marked this pull request as ready for review
July 22, 2026 15:54
The account daemon legitimately outlives its last client for a moment while it drains, holding its log and config DB open. POSIX rm does not care, but on Windows an open file blocks deletion, so the caller's cleanup of the smoke cache raced the daemon's asynchronous shutdown and failed with "Device or resource busy" after every phase had passed — first visible on windows-latest once the agent-config phases stopped failing earlier, and reproduced identically on the local Windows VM. The smoke now ends with an explicit phase: stop the daemon through its own lifecycle command, wait (bounded) until status reports not-running, and give Windows one beat for the final handle close. A daemon that will not retire is now a smoke FAILURE in its own right instead of a cleanup accident. Verified red-to-green on the VM and green on macOS. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The first full Windows x64 CI run of this branch (earlier contract-step walls had always aborted the leg) surfaced 40 failures: the managed launcher's fail-closed install/update/uninstall gates and the suite's POSIX-modeled activation tests were both new in this branch and had never been reconciled, and they hid three real production bugs. Production fixes: - ui: config publishes with POSIX rename semantics. MoveFileExW's legacy replace cannot supersede a destination a reader holds open (the name lingers until the last handle closes, even with FILE_SHARE_DELETE), so a config save racing an open reader failed. The temp file is opened with DELETE access and renamed in-hand (bare-path target, NUL-padded rename record) before close, falling back to MoveFileExW on any rename-by-handle failure so non-NTFS config homes keep the previous behavior. - subprocess(win): the child's redirected log handle now shares write access. POSIX places no lock on the redirected log, so a second producer may append while the child runs; the missing FILE_SHARE_WRITE mandatory-locked every other writer out, which the index-supervisor callback-isolation test legitimately exercises. - cli(win): the portable uninstall path composes the managed binary name with its .exe suffix; it could never find (or remove) the installed binary before. - activation: security-predicate refusals are now diagnosable. The transaction's predicates refuse without a usable OS last-error, which collapsed five distinct refusal classes into one blind "status -3, os 0" at every caller. Each refusing predicate records what refused — predicate, untrusted SID, and object path — first-wins so the deepest cause survives, and the staging error line appends it. This is what identified every Windows refusal in this batch, including the real install-failure class on drive-root-rooted payload directories (Authenticated Users holds inherited Modify there; profile-rooted directories do not). Windows test dispatch (release binaries unchanged): - Test builds (CBM_CLI_ENABLE_TEST_API, the existing test-object macro) compile the portable activation flow on Windows, reachable solely through the installed activation test seam and announced loudly on stderr; release binaries compile exclusively the managed transaction, so the dispatch polarity fails closed. The suite's install/update/ uninstall calls route through a seam-if-absent wrapper, keeping the shared activation and agent-config semantics unit-covered on Windows; POSIX arms keep exercising the real activation machinery. - Two new release-gate tests assert the fail-closed contract with the seam OFF: a portable payload may not self-update/self-uninstall, and install refuses to adopt or replace a foreign file at the canonical launcher path even under --force. - Test-bug fixes the first valid Windows runs exposed: the version- cohort crash test reused one conflict record across the probe and the turnover acquire (whose entry memset erased the probe's detail before the assertions read it); the activation staged-path assertion now checks containment rather than a separator (the \\?\ namespace performs no forward-slash translation); the ui atomic test captures, cleans up, then asserts so a failure cannot poison later tests; the PATH-visibility guard is POSIX-only (Windows configures PATH through the user registry, not a shell rc). Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…librated suite tier The 40 hidden Windows failures fixed in the previous commit reached CI unseen because the local Windows leg never validly ran the C suite: MSYS2's shared /tmp fails the secure-ancestry validation (refusals, not signal) and the ssh driver piped everything through `tail -40`. This closes those holes and hardens the gates that let an install failure scroll past. - vm-run-tests.sh (new, used by win.sh test / ubsan-test / trap-ubsan-test): gives the suites CI's exact protected per-user temp root (owner-stamped, protected current-SID DACL, mirroring the workflow's pwsh step), shapes the runner's build directory like a real user checkout (drive-root trees inherit an Authenticated-Users Modify ACE that profile-rooted checkouts do not have, which the activation source-directory policy correctly refuses; DACL re-rooted two-step — directory protect then child /reset, since inheritance flags are directory-only and a /T re-root leaves files with empty deny-all DACLs), streams the FULL output, and refuses to report success unless the runner printed its completion summary. - smoke: Phase 8 no longer swallows the install exit code. On Windows it now requires the managed install to SUCCEED and to publish an exact two-link canonical launcher — a staging refusal used to scroll past as tolerated noise while the downstream config assertions kept passing. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…erywhere The incremental suite cloned fastapi from the network on every run and indexed a DIFFERENT corpus per environment: CI used a sparse checkout without docs/ and tests/ (the assertion thresholds are sized for it) while local runs indexed the full tree — twice the files for the same assertions, a per-run network dependency, and the suite's dominant cost on the sanitized container leg. The fixture is now cloned from the network at most once per machine into a persistent cache (committed with an atomic rename so a torn download can never pose as a valid cache; CBM_TEST_FIXTURE_CACHE overrides the location), every run local-clones from it offline, and docs/ plus tests/ are trimmed everywhere — the portable equivalent of CI's historical sparse profile, so all environments now index the same corpus the assertions were written against. Same 163 tests, same assertions: 74s -> 31s native, with the container leg's cut scaling with the halved indexing volume, and the "network offline" flake source reduced to first-run-per-machine. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
… caches PR iteration was bounded by single-job suite runs (~40-50 min critical path) and by every first-of-ref build starting cold. Same suites, same sanitizers, same gates — the schedule, the cache reuse, and a new runtime coverage proof change. - run-tests-parallel.sh accepts CBM_TEST_SHARD="i/N" and runs a deterministic slice of --list-suites; unset selects everything, so the path is inert outside CI. Known-heavy suites are dealt to shards in weight order first (naive modulo stacked the two slowest, store_arch and daemon_runtime, onto one job), the parallel wave and the serial tail are sliced separately so every shard keeps its own quiet tail, and the per-shard union guard proves each job ran exactly its slice. - shard-completeness job: per-shard guards cannot see a mis-plumbed CBM_TEST_SHARD (two jobs running the same slice passes every local check while a slice runs nowhere). Every leg now uploads a manifest (leg, i/N, sha of the full suite list, its slice) and one aggregation job re-proves per leg that the shards agreed on the list, indices are exactly 1..N, and the union of slices IS the list. Runs on unsharded topologies too, where each leg's single manifest must cover the list. - workflow topology (shard_suites input, off by default = byte-identical to the pre-shard matrix): ubuntu legs run 3 shards, Windows 2 (every extra Windows shard re-pays ~5 min of MSYS2 setup), macOS stays at 1 (not the critical path; mac runner concurrency ceilings are the tightest). The Windows test job also pre-creates build\c with a protected DACL so build products inherit it — workspace drive roots grant Authenticated Users Modify by inheritance, which the activation transaction's source-directory policy correctly refuses. - ccache: a final ref-less restore key lets a fresh PR start from the newest cache GitHub's scoping permits, and a nightly build-only cache-warm workflow keeps main-scoped caches at most a day stale — without it the fallback had no warm source, since nothing built in main's scope. The strictly-per-ref policy this replaces cost ~8-12 minutes on every first-of-ref build for no safety gain: CCACHE_COMPILERCHECK=content makes a stale or foreign cache able to miss but never to return wrong output. Sharded legs restore-all/ save-one (every shard builds the identical objects, so shard 1's cache carries the full set) — save volume stays flat and warm caches stop being evicted by per-shard duplicates. - suite timeouts: daemon_runtime joins the slow tier. It measures ~610s solo on arm64 under ASan (10-round loop, no hang), so the 900s default under a loaded 4-job CI runner was a slowness kill masquerading as a hang detector. Validated locally: 3-shard union over the real suite list is complete with no duplicates, one top-heavy suite per shard, and the summed 3-shard totals reproduce the unsharded run exactly (6776 passed, 0 failed, 4 skipped). Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Flip shard_suites on for all three callers of the reusable test workflow. This commit is deliberately the only topology change: reverting it alone restores the previous single-job-per-leg CI shape while keeping the (inert without it) sharding machinery and the cache-warm pipeline. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…ainer I/O off virtiofs Local runs left most of the machine idle: the serial tail ran sixteen suites one at a time on otherwise-idle cores, the three platform legs were launched by hand (usually sequentially), and every container-leg object write crossed the virtiofs bind mount. - run-tests-parallel.sh tail scheduling in two phases: the FLEX suites (timing-shaped but free of the shared per-account daemon runtime namespace) run CBM_TAIL_JOBS-wide (default 2), then the EXCL group — daemon-family plus the suites that drive daemon one-shots or supervisor rendezvous — runs strictly sequentially on a machine exactly as quiet as the old fully-serial tail gave it. The wave was already fed longest-first by the shard dealing order, so the drain-out no longer ends on a heavy straggler. - ladder.sh: one maintained entry point for the full local push gate with the legs overlapped — lint, the Linux container suite, and the Windows VM suite in the background, the macOS suite in the foreground, one verdict per leg, logs kept per leg. A missing prerequisite fails its leg loudly instead of silently skipping. - win.sh test-par now runs through vm-run-tests.sh (--par mode): the full parallel harness under the CI-shaped protected temp root with complete output. It previously ran under the MSYS-shared /tmp and piped through `tail -25` — the same truncated-blindness class that hid 40 Windows failures from the `test` command. - docker-compose: build artifacts and the incremental fixture cache move to named volumes on the container VM's native filesystem. Object writes over the virtiofs bind mount are the container legs' largest avoidable I/O cost, and the fixture cache now survives across container runs. - test_mem(win): the first valid full-parallel VM run proved working-set trimming beats the re-touch mitigation (19 MB resident of a 256 MB double-touch at 18 parallel suites). The RSS probe now VirtualLocks a 64 MB span — locked pages are exempt from trimming, making the measurement pressure-immune — with bounded touch-and-sample retries when the lock is unavailable. Red-to-green under the same 18-job load. - cli: the portable install's staging error now appends the activation refusal note (predicate, SID, object) like the managed path already does — a bare "activation transaction I/O failed" on a CI-only failure is undiagnosable without it. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The workflow pre-created build\c with a protected DACL so build products would inherit it, but the diagnostic note from the failing CI shard named the real actor: MSYS2's Cygwin layer writes POSIX-emulating DACLs — including a CREATOR OWNER (S-1-3-0) mutation grant — onto directories its tools touch, overwriting the pre-created shape during the build. The activation transaction's source-directory policy correctly refuses that grant, so the install-flow tests failed on CI while the VM (whose harness stamps AFTER building) stayed green. The stamp now runs inside run-tests-parallel.sh on Windows, after any builder has had its say and immediately before the suites — one code path for CI and the VM. Two idempotent steps, as before: protect the directory, then /reset the children onto the clean inherited set. Verified: full VM parallel run green (6669 passed, 0 failed) through the new path. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Phase 9b's fixture installs spawn ephemeral daemons whose mapped generation backings pin files in the fixture HOME; the cleanup rm then races the daemon's asynchronous drain and loses nondeterministically on Windows (Device or resource busy on a generations backing — the same class the final-phase daemon retirement already fixed for the smoke cache, two rounds green then one red on identical code). The retirement is now a shared helper with a cheap not-running probe, called before every fixture cleanup that follows an install and by the final phase; a daemon that will not retire stays a hard failure. The 9b fixtures also route LOCALAPPDATA under the fixture HOME: their installs staged launcher pairs into the real runner profile, outside the fixture's isolation and the harness's cleanup reach. Verified: full VM smoke green end to end (all 9b subphases, phase 17 through the shared helper). Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Open
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds one mandatory per-account coordination daemon for long-lived MCP and hook sessions.
Coordination and activation
All concurrent CBM processes must agree on the exact version, executable build, coordination ABI, and canonical
CBM_CACHE_DIR. Conflicting processes fail before doing work, report the conflict to the launching session, and record it indaemon-conflicts.ndjson.Native
install,update, anduninstallcommands:Windows uses a permanent launcher with an atomically replaceable payload, allowing the same lifecycle guarantees despite executable-file locking. npm, PyPI, and Go wrappers verify and publish their private cached binaries safely.
Security and observability
Verification
Recent local verification on the merged checkpoint line:
Current-head GitHub CI is intentionally left running on this draft PR.
Handoff / where work stopped (2026-07-18)
Pushed checkpoint:
7696b1486a10f19e231f5b8285abe6eea71b9a3c.The latest checkpoint additionally:
/usr/bin/true, which is policy-dependent on older macOS runners;mainand its isolated parallel-suite runner.Known next task:
TEMP,TMP,TMPDIR, runtime/cache paths, and the MSYS/tmpmapping through it for the full Windows harness. Re-run exact-head Windows smoke/guards/native tests afterward.No merge was performed; the PR remains draft for the follow-up session.